Victora et al. BMC Public Health 2013, 13(Suppl 3):S24 
http://www.biomedcentral.com/1471-2458/13/S3/S24 



(bmc 



Public Health 



REVIEW Open Access 



How within-country inequalities and co-coverage 
may affect LiST estimates of lives saved by 
scaling up interventions 

Cesar G Victora 1,2 *, Aluisio J D Barros 1 , Tanya Malpica-Llanos 2 , Neff Walker 2 



Abstract 

Lives-saved estimates calculated by LiST include the implicit assumptions that there are no inequalities among 
different socioeconomic groups, and also that the likelihood of a mother or child receiving a given intervention is 
independent from the probability of receiving any other interventions. It is reasonable to assume that, as a 
consequence of these assumptions, LiST estimates may exaggerate the numbers of lives saved in a population, by 
ignoring the fact that coverage is likely to be lower and mortality higher among the poor than the rich, and also 
by failing to take into account that coverage with different interventions may be clustered at individual mothers 
and children - a phenomenon described as co-coverage. We used data from 127 DHS surveys to estimate how 
much these two assumptions may bias estimates produced by LiST, and conclude that under real-life conditions 
bias occurred in both directions, with LiST results either over or underestimating the more complex estimates. With 
few exceptions, bias tended to be small (less than 10% in either direction). 



Introduction 

The Lives Saved Tool, or LiST (http://www.jhsph.edu/ 
departments/international-health/IIP/list/) is being 
increasingly used to estimate the impact of changes in 
intervention coverage on the mortality of children under 
five years of age [1-3]. LiST is a computer-based tool 
that estimates the impact of scaling up interventions on 
maternal, neonatal and child mortality. The main pur- 
pose of this tool is to promote evidence-based program 
planning for maternal, neonatal, and child health [4]. 
The model works by establishing a baseline description 
of the country or region in terms of demographic infor- 
mation, nutritional status, causes of deaths and levels of 
risk factors and exposure variables (levels of intrauterine 
growth restriction, breastfeeding patterns and rates by 
age, percent of the population exposed to falciparum), 
and current coverage of over 60 interventions. Each 
intervention is also defined in terms of its efficacy in 
reducing one or more causes of death. The model allows 
users to create various scenarios where they can scale 
up different intervention packages and estimate the 
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impact of the various scenarios of maternal, neonatal 
and child mortality. 

There have been numerous comparisons to the mod- 
eled impact from LiST to measured mortality reduction 
(e.g., [1-3,5]). Overall, these comparisons suggest that in 
general LiST can be used to estimate the impact of scal- 
ing up interventions on neonatal and child mortality. In 
this article we explore how two separate issues may 
affect the validity of national LiST estimates. 

First, the algorithm used in LiST to estimate mortality 
ignores the fact that children who are most likely to die 
(e.g. those from the poorest families) may also be less 
likely to receive life-saving interventions [6]. If interven- 
tion coverage increases more rapidly among the rich, who 
were less likely to die at baseline, the impact estimated by 
LiST may be biased upwards. We will investigate this by 
using household asset indices to stratify national survey 
samples into wealth quintiles, and then run LiST sepa- 
rately in each quintile for selected countries. We will cal- 
culate the number of deaths prevented by scaling up 
selected interventions using two approaches: (a) assuming 
that they are equally scaled up in all quintiles (as LiST cur- 
rently does) and (b) carrying out separate simulations for 
each quintile using actual increases in coverage, and sub- 
sequently aggregating the number of deaths prevented in 



© 201 3 Victora et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons 
BiolVlGCl CGntrBl Attribution License (http://creativecommons.Org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in 
any medium, provided the original work is properly cited. 



Victora et al. BMC Public Health 2013, 13(Suppl 3):S24 
http://www.biomedcentral.com/1471-2458/13/S3/S24 



Page 2 of 7 



the whole population. The comparison of the two sets of 
results will allow us to estimate how much bias may arise 
from ignoring within-country inequalities. 

The second issue is that LiST assumes independent 
probabilities of being covered by different interventions; 
for example, a mother who receives antenatal care is just 
as likely to have her child delivered by a skilled attendant 
as a mother who did not receive antenatal care. This is 
unlikely to be true in the real world, where interventions 
will tend to be clustered at the mother/ child level, a find- 
ing that has been described as "co-coverage" [7]. Lack of 
independence among several interventions is likely to lead 
to an overestimate of the number of lives saved, given that 
the increase in coverage may be concentrated on children 
already receiving other interventions and under a lower 
mortality risk. We will perform simulations using existing 
datasets to assess the magnitude of co-coverage and how 
this may affect LiST estimates. 

We tested these two potential sources of bias using 
data from the Countdown to 2015 Equity Database 
[8,9]. To our knowledge this is the first examination of 
these issues in the literature. 

Methods 

Different data and approaches were used in the two sets 
of analyses. 

Effect of wealth inequalities 

We listed all Demographic Health Surveys (DHS) and 
Multiple Indicator Cluster Surveys (MICS) carried out in 
Countdown countries since 1990 that included data on 
household assets. Thirty five countries had two or more 
surveys with an interval of 5-13 years between them. If a 
country had only two such surveys, we selected these 
two. If it had three or more, we gave priority to selecting 
two surveys of the same type (i.e. two DHS or two MICS) 
in order to increase comparability of results. If there were 
three surveys of the same type, we chose the earliest and 
the most recent, in order to maximize the number of 
years between them. Of the 35 countries, 23 were in sub- 
Saharan Africa, eight in Asia, three in Latin America and 
the Caribbean and one in North Africa. This database 
has been recently used in analyses of changes in inequal- 
ities over time [10]. 

We selected two interventions, each of which is effective 
against a single cause of death. We avoided interventions 
that are effective against several causes of death, such as 
antenatal care, skilled birth attendant or breastfeeding, 
because these are more difficult to model and involve a 
larger number of assumptions. The first intervention, 
care-seeking for pneumonia, has been present for a long 
time and shows important inequalities in most countries 
[11]. The second represents a recently introduced inter- 
vention, insecticide treated net use by children (ITNs) in 



malaria-endemic countries, for which coverage is increas- 
ing rapidly in many countries. 

A household-level wealth index was used to stratify 
mothers and children according to socioeconomic posi- 
tion, as calculated by the original DHS or MICS survey 
[12]. The wealth index was divided into quintiles, with 
Ql representing the poorest 20% of households and Q5 
representing the richest. Individuals were classified 
according to the quintile of the household where they 
lived. 

To ensure full compatibility between the quintile- 
specific and national estimates of mortality and cover- 
age, we used for the latter the weighted average of the 
quintile-specific levels, instead of the aggregate values 
obtained from the national analyses. The differences 
between the two sets of national values were very small, 
less than 2 percent points for the large majority of 
comparisons. 

We did six separate LiST runs, one for each quintile and 
one for the whole population. We then summed the num- 
ber of lives saved in the five quintiles to obtain the adjusted 
estimate. Then, this adjusted estimate was compared to the 
aggregate estimate (ignoring the quintiles) using a bias 
indicator calculated as follows: 

No. lives saved using traditional LiST estimates 

Relative bias = 

No. lives saved using wealth — stratified LiST estimates 

This approach assumes that the equity-stratified estimates 
are the true values, and differences between these and the 
traditional LiST estimates constitute bias. If bias was greater 
than one, traditional analyses are over-estimates. If smaller 
than one, underestimates. In the table below, we express 
bias as a percentage, after subtracting 1 from the above 
formula. 

To examine the magnitude of bias resulting from 
ignoring wealth quintile differences when performing 
LiST estimates, we used the following parameters for 
the 35 countries: 

1. We obtained estimates of intervention coverage by 
quintile, for the baseline and endline surveys. 

2. We obtained estimates of under-five mortality by 
quintile from the baseline survey. 

3. We obtained the number of births by quintile, also 
from the baseline survey (quintiles were calculated for 
households, and due to higher fertility rates, usually 
there are more mothers and children in the poorer 
quintiles than in the richer quintiles). 

4. We used country-specific proportions of deaths due 
to pneumonia and malaria, as calculated by the Child 
Health Epidemiology Reference Group [13]. 

5. We used the efficacy data included in the LiST soft- 
ware to estimate lives saved by pneumonia care-seeking 
[14] and ITNs [15]. 
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6. Slope indices of inequality were calculated for cov- 
erage change over time. These express the absolute dif- 
ference, in percent points (p.p.), between subjects at the 
top and bottom of the wealth scale [16]. Positive values 
show that coverage increases were larger among the rich 
than for the poor. 

As a consistency check, we followed exactly the same 
procedures described above using an Excel spreadsheet, 
and compared the results with those obtained with 
LiST. 

Effect of co-coverage 

We analyzed data from 127 DHS surveys from 60 coun- 
tries, that were available by early 2012. A full list of sur- 
veys and countries is available in Additional file 1. 
Within each survey, the units of analyses were mother- 
child pairs, and we calculated correlation coefficients for 
all possible combinations of six coverage indicators, all 
coded as 0 or 1: 

- ANC: antenatal care (4 or more visits), 

- SBA: skilled birth attendant, 

- DPT: three doses of diphtheria, pertussis and tetanus 
vaccine, 

- WAT: improved water source, 

- VTA: vitamin A supplementation in the previous six 
months, and 

- ITN: insecticide treated net use by the child. 

We used the results from the correlation analyses to run 
a series of simulations using Excel. The simulations were 
carried out with several assumptions. We set mortality 
due to causes preventable by ANC or SBA at 50 per thou- 
sand births. For simplicity we set baseline coverage levels 
with both interventions to 0%, and increased it to 100%. 
We set efficacy of ANC at 0.2 (20% reduction in mortality 
with universal coverage) and SBA efficacy at 0.3, and their 
joint efficacy at 0.44, that is, 20% of deaths would be pre- 
vented by ANC and of the remaining 80% of deaths, 30% 
(or 24% of the total number of deaths) would be prevented 
by SBA, if both interventions were scaled up to 100% cov- 
erage. This modeling exercise assumes that there is neither 
synergism nor antagonism between SBA and ANC. It 
should be noted that baseline coverage, mortality levels 
and efficacy parameters do not affect the calculation of 
bias because these are applied across the board, both in 
the traditional LiST approach and in the estimates that 
allow for co-coverage. 

In the traditional LiST approach, we assumed inde- 
pendence between SBA and ANC coverage, that is, 
mothers receiving ANC were as likely to receive SBA as 
those mothers who did not receive ANC. We then esti- 
mated how many lives would be saved by scaling up 
coverage to 100% with both interventions. 

The co-coverage approach assumed different degrees 
of co-coverage, that is, mothers using ANC were more 



likely to also receive SBA. This was done by varying the 
conditional probability of receiving SBA according to 
whether or not the mother had also received ANC, so 
as to reproduce different magnitudes of the correlation 
coefficient between the two variables. The population 
was then divided into the four possible combinations of 
ANC and SBA, the numbers in these subgroups were 
estimated on the basis of the conditional probabilities, 
and the above-described efficacy levels were applied to 
the four subgroups of the population, allowing us to 
estimate how many lives would be saved in each 
subgroup. 

Consistently with the equity analyses, bias was defined 
as: 

No. lives saved using traditional LiST estimates 

Relative bias 

No. lives saved using co - coverage adjusted LiST estimates 

Again, if bias was greater than one, traditional analyses 
are over-estimates, if smaller than one, underestimates. 
In the table, we express bias as a percentage, after sub- 
tracting 1 from the above formula. These simulations 
were carried out using Excel because LiST does not cur- 
rently allow for lack of independence between coverage 
with separate interventions. 

Results 

Effect of wealth inequalities 

Of the 35 countries with two surveys, 34 had data on 
careseeking for pneumonia. Fourteen of these showed 
either declines in coverage over time or increases of less 
than three percent points, and were excluded from the 
analyses. Results for the remaining 20 countries are pre- 
sented in Table 1. The equity profiles for all countries 
with data on careseeking for pneumonia and insecticide- 
treated bednet are presented in Additional file 1. 

The average bias in the 20 countries was -0.6% and 
the median was -1.6%, indicating that the traditional 
approach was very similar to the equity-stratified 
approach overall. Bias above 10% was observed in six 
countries. The most extreme case was Madagascar, 
where bias was equal to 63%. This was a result of 
declines in coverage in the two poorest quintiles, in con- 
trast with increases in the three richest quintiles. Nega- 
tive bias, showing underestimation by LiST, was also 
present in some countries, with Bolivia being the most 
extreme example with a value of -22% that was asso- 
ciated with faster increase in coverage among the poor 
than for the rich. 

Table 1 also shows the slope index of inequality. Positive 
values indicate that coverage increases were larger for the 
rich than for the poor, and negative values show the oppo- 
site trend. In general, countries with positive bias also had 
positive slope indices, with a Spearman correlation of 0.81 



Victora et al. BMC Public Health 2013, 13(Suppl 3):S24 
http://www.biomedcentral.com/1471-2458/13/S3/S24 



Page 4 of 7 



Table 1 Bias in LiST estimates resulting from ignoring within country inequalities, in 20 countries with increases over 
time in care-seeking for pneumonia. 
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Niger 


1998 


2006 


26% 
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22 


-16 


-2% 


Uganda 
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2006 


61% 
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12 


-31 


-3% 


Lesotho 


2004 


2009 


59% 


66% 


7 


-AA 


-6% 


Mali 


1995 


2006 


32% 


38% 


6 


-20 


-6% 


Kyrgyzstan 


1997 


2005 


48% 


63% 


15 


-70 


-7% 


Peru 


1996 


2004 


46% 


71% 


25 


-18 


-9% 


Egypt 


1995 


2008 


62% 


73% 


12 


-21 


-12% 


Benin 


1996 


2006 


32% 


36% 


A 


-17 


-15% 


India 


1998 


2005 


64% 


70% 


6 


-18 


-16% 


Bolivia 


1998 


2008 


43% 


51% 


8 


-20 


-22% 



(*) Slope index of inequality for change in coverage by wealth quintile, expressing the difference in percentage points between the increase in coverage for those at the 
top of the wealth distribution compared to those at its bottom. 



(p<0.001) between the two variables. Bias, on the other 
hand, was not associated with overall coverage change 
(Spearman r = 0.18; p=0.45). 

We carried out similar analyses for ITNs. Of the 21 
malaria-endemic countries with ITN data, four had 
increases in ITN lower than 3 p.p. between the two sur- 
veys; in three other countries, malaria accounted for 
fewer than 3% of all deaths. Both groups of countries 
provided highly imprecise estimates of bias, and were 
excluded from the analyses. Data for the remaining 14 
countries are presented in Table 2. 

The average bias in the 20 countries was 1.4% and the 
median was 0.1%, indicating that the traditional approach 
was very similar to the equity-stratified approach overall. 
Bias above 10% was observed only in the Central African 
Republic and in Cote d'lvoire; the traditional LiST 
approach overestimated impact on both countries. Nega- 
tive bias, showing underestimation by LiST, was small 
when present. 

Correlations between bias and overall coverage 
increase (r=-0.393; p = 0.17) and the slope index of cov- 
erage change (r=0.294; p=0.31) were not significant. 

Effect of co-coverage 

Table 3 shows the mean values of the correlation coeffi- 
cients across all 127 surveys with data on the relevant 
indicators, as well as their variability. Nearly all surveys 
had data on ANC, SBA, DPT and WAT. Information on 



VTA was available for 59 surveys, but only 18 malaria- 
endemic countries had data on ITN. 

Most correlation coefficients were positive, as is 
shown by their mean values. However, at least one 
country had negative correlations for all possible combi- 
nations of coverage indicators, as shown by the fact that 
minimum values were negative. 

The strongest mean correlation was between ANC 
and SBA, which is hardly surprising because these are 
often delivered at the same location by the same health 
workers. However, even for this combination the mean 
correlation coefficient was relatively weak (r=0.322). All 
other mean correlation values were below 0.2, indicating 
weak correlations. 

The next step was to assess how much bias would be 
introduced by ignoring co-coverage in LiST, given dif- 
ferent strengths of the correlation between any two vari- 
ables, and different coverage levels. 

Table 4 shows different scenarios with variable levels of 
co-coverage. We use SBA and ANC as examples, but the 
findings apply to any combination of two coverage indica- 
tors. SBA coverage was set to 20%, 40%, 60% and 80%. We 
then varied the level of co-coverage to reproduce three 
scenarios: weak (r = 0.1), moderate (r = 0.3) and strong (r 
= 0.5). In the 127 surveys, the mean value for ANC cover- 
age was 56% and the mean correlation between ANC and 
SBA was 0.322 (range -0.016 to 0.612). We then modeled 
different levels of co-coverage - as assessed by Pearson 
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Table 2 Bias in LiST estimates resulting from ignoring within country inequalities, in 20 countries with increases over 
time in care-seeking for pneumonia. 



Country 


First survey 


Second survey 


Coverage change 


Slope index* 


Bias 


CAR 


1994 


2006 


15 


28 


15% 


Cote Ivoire 


1994 


2006 


■-1 


7 


13% 


Senegal 


1997 


2005 


7 


2 


3% 


Benin 


1996 


2006 


21 


30 


1% 


Zambia 


1996 


2007 


28 


10 


1% 


Niger 


1998 


2006 


7 


8 


1% 


Ghana 


1998 


2008 


29 


-3 


0% 


Tanzania 


1999 


2010 


61 


1 


0% 


Mali 


1995 


2006 


27 


7 


-1% 


Togo 


1998 


2006 


39 


-8 


-1% 


Cameroon 


1998 


2006 


13 


10 


-2% 


Malawi 


2000 


2010 


40 


27 


-2% 


Madagascar 


1997 


2008 


45 


A 


-3% 


Uganda 


1995 


2006 


10 


3 


-3% 



correlation coefficients - between ANC and SBA, for dif- 
ferent coverage levels with SBA (Table 4), and calculated 
the amount of bias that would result as a consequence of 
ignoring co-coverage. 

Contrary to the original expectation, ignoring co-cov- 
erage often resulted in underestimation of the number 
of lives saved, rather than in overestimation. This is 
because - particularly when SBA coverage was high - 
moderate or strong co-coverage led to a high proportion 
of the population receiving both SBA and ANC and 
therefore to more lives being saved in the population as 
a whole than expected in the analyses where co-cover- 
age was not modeled. That is, co-coverage per se dis- 
placed a substantial proportion of the population to the 



group receiving both interventions, particularly when 
overall SBA coverage was high. 

In any case, even under these rather extreme assump- 
tions, bias was consistently less than 10.2%. 

Discussion 

To our knowledge, this is the first examination of how 
LiST estimates may be affected by socioeconomic 
inequalities in coverage and mortality, and by the fact 
that interventions tend to be clustered in the same 
children. 

Our original assumptions were that there would a 
substantial amount of bias in LiST estimates resulting 
from ignoring within-population inequalities. In general, 



Table 3 Mean values of Pearson correlation coefficients for selected pairs of interventions, based on individual 
(mother-child) level analyses in 127 DHS surveys. 



Correlation 


Surveys 




Pearson correlation coefficients (r) 








Mean 


Standard deviation 


Minimum 


Maximum 


SBA X ANC 


127 


0.322 


0.130 


-0.016 


0.612 


SBA X DPT 


124 


0.191 


0.114 


-0.058 


0.526 


SBA X WAT 


126 


0.126 


0.105 


-0.234 


0.416 


SBA X VTA 


59 


0.091 


0.085 


-0.057 


0.397 


SBA X ITN 


18 


0.078 


0.068 


-0.024 


0.218 


ANC X DPT 


125 


0.191 


0.109 


-0.081 


0.469 


ANC X WAT 


126 


0.088 


0.081 


-0.126 


0.408 


ANC X VTA 


59 


0.090 


0.082 


-0.042 


0.395 


ANC X ITN 


18 


0.058 


0.039 


-0.022 


0.126 


DPT X WAT 


124 


0.071 


0.078 


-0.258 


0.316 


DPT X VTA 


59 


0.147 


0.111 


-0.063 


0.390 


DPT X ITN 


18 


0.044 


0.052 


-0.044 


0.144 


WAT X VTA 


58 


0.037 


0.047 


-0.074 


0.175 


WAT X ITN 


18 


0.045 


0.046 


-0.081 


0.113 


VTA X ITN 


17 


0.040 


0.027 


-0.021 


0.081 
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Table 4 Simulations of bias associated with different 
strengths of co-coverage, using SBA and ANC as an 
example. 



SBA coverage 



20% 40% 


60% 


80% 


Pearson r 


Scenario 1: No co-coverage 








Lives saved 7700 1 0400 


13100 


15800 


0.00 


Scenario 2: Weak co-coverage 








Lives saved 7840 10680 


13520 


16360 


0.08 to 0.10 


Bias -1.8% -2.6% 


-3.1% 


-3.4% 




Scenario 3: Moderate co-coverage 








Lives saved 7180 10360 


13540 


16720 


0.24 to 0.30 


Bias 7.2% 0.4% 


-3.2% 


-5.5% 




Scenario 4: Strong co-coverage 








Lives saved 6990 10480 


13970 


17460 


0.40 to 0.51 


Bias 10.2% -0.8% 


-6.2% 


-9.5% 





we expected LiST to overestimate the number of lives 
saved, because intervention coverage would increase fas- 
ter among the rich, whereas most deaths occur among 
the poor. 

We selected two interventions for analyses, each of 
which prevented deaths due to a single cause. We avoided 
complex interventions, such as ANC, SBA or breastfeed- 
ing, which provide protection against multiple causes and 
therefore involve more complex modeling. Care-seeking 
for pneumonia has been promoted for many years, shows 
reasonably high coverage in many countries, and tends to 
be quite inequitable. ITN is a low-coverage, recent inter- 
vention that is reasonably equitable [11]. 

Our preliminary conclusions are sobering. Bias went in 
both directions, and on average across all countries there 
was virtually none. Bias tended to be largest for pneumo- 
nia careseeking than for ITN, as expected because the for- 
mer is more inequitably distributed within populations. 
The most extreme values in either direction were lower 
than 30%, either over or underestimation. 

LiST includes complex calculations that take into 
account baseline mortality levels, baseline coverage, and 
change in coverage over time. Inequalities in each of these 
parameters could theoretically affect how much traditional 
LiST results might over or underestimate the true number 
of lives saved, obtained with equity-stratified calculations. 
Nevertheless, our results do not suggest that baseline cov- 
erage or coverage change over time are strongly associated 
with the amount of bias, which in general was small. Our 
recently published analyses of coverage change over time 
by wealth quintile in countries with two consecutive sur- 
veys [17] showed that coverage tended to increase more 
rapidly in the poor that in the rich over time, in most 
countries. Our original expectation was that coverage 
would increase faster among the rich, and thus lead to 
bias. Because the same dataset was used in the present 



analyses, this may explain why bias was not marked in 
most countries. 

The second issue we addressed was that ignoring co- 
coverage would also bias LiST estimates. Because current 
versions of LiST do not allow for taking co-coverage into 
account, a series of simulations were performed with 
Excel. The assumption behind these analyses was that, 
because interventions will tend to be clustered at the level 
of individual mothers and children, the traditional LiST 
approach that assumes independence among interventions 
would overestimate the number of lives saved. Our ana- 
lyses of 127 DHS surveys show that in fact most interven- 
tions show positive correlations at the mother/child level, 
but that correlation coefficients tend to be weak or moder- 
ate. The strongest average correlation across all combina- 
tions of six interventions was 0.32, for ANC and SBA, 
both of which are delivered by the same health workers in 
the same facilities. Our simulation exercise showed that 
this level of intensity for co-coverage will only result in 
small amounts of bias, typically of 10% or less. Contrary to 
the initial expectations, ignoring co-coverage could result 
in bias in either direction. In particular, high coverage 
with one intervention in the presence of moderate or 
strong co-coverage will displace a larger proportion of the 
population to the group covered by both interventions, 
and therefore save more lives than would be expected in 
the analyses that ignore co-coverage. 

There are theoretical reasons for assuming that the 
traditional LiST approach would be biased due to ignor- 
ing within-country inequalities and co-coverage. How- 
ever, the presence exercise with real data from all 
Countdown to 2015 countries with two consecutive sur- 
veys shows that such bias is unlikely to be important in 
most countries. 

Taken together, the results of the present analyses 
suggest that neither the fact that the traditional LiST 
approach ignores within population inequalities, nor 
that it ignores co-coverage, are likely to lead to impor- 
tant errors in the resulting estimates in the vast majority 
of countries. 

Additional material 



Additional file 1: The additional contains figures showing the 
observed inequalities for careseeking for pneumonia and 
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